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ABSTRACT 



An adaptive vector quantization process and quantizer (VQ) 
using a clustering technique known as AFLC (adaptive 
fuzzy leader clustering) is disclosed. The quantizer, AFLC- 
VQ, has been designed to vector quantize wavelet decom- 
posed sub images with optimal bit allocation. The high- 
resolution sub images at each level have been statistically 
analyzed to conform to generalized Gaussian probability 
distributions by selecting the optimal number of filter taps. 
The adaptive characteristics of AFLC- VQ stem from using 
self-organizing neural networks with fuzzy membership 
values of the input samples for upgrading the cluster cen- 
troids based on well known optimization criteria. The cen- 
troid of each pattern group can represent all the other 
patterns in that cluster. The entire data set can be indexed 
into look up tables sent to the user for decoding and viewing 
the image. By generating look up tables or codebooks and 
entropy coding of the quantizer output, AFLC-VQ can 
compress images at 100:1 or greater for transmitting while 
reconstructing the images with high fidelity. 

17 Claims, 8 Drawing Sheets 
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VQ parameters for Visible Human Image (a_vm1480, 2048x1024, red plane) 
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FIG. 2a 



VQ porometers for Visible Human Image (a_vm1480, 2048x1024, green plane) 
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FIG. 2b 



VQ parameters for Visible Human Image (a_vm1480, 2048x1024, blue plane) 
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Vector Quantization paramaters for Cervical Image (512 x 512) 
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FIG. 4b 
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Comparison between AFLC-VQ, JPEG and EZW (Cervical Spine Image) 
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Comparison between AFLC-VQ, JPEG and EZW (visible Human Image) 
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FIG. 7 
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ADAPTIVE VECTOR QUANTIZATION/ clustering of the data, subsequent transmission or storage of 

QUANTIZER that compressed data, followed by reconstruction of the 

original data. 

CROSS-REFERENCE TO RELATED it is another object of the present invention to provide a 

APPLICATION 5 process for fast high fidelity transmission of images. 

Priority is claimed under 35 U.S.C. §119 (e) on the SUMMARY OF THE INVENTION 

Provisional Application, Serial No. 60/136,418, filed on ™ . i_- - ^ , 

May 28, 1999, of common title, of common inventorship The above objects are satisfied and other lmiitaUons of the 

and assignment with the present application. 1fl P nor H 1 arc ™**>™ m present invention. The present 

10 invention includes m combination an Adaptive Fuzzy 

FIELD OF THE INVENTION Leader Clustering or AFLC for on-line generation of code 

vectors, and applying AFLC to quantization of wavelet 

The present invention relates generally to data transformed sub images. The present invention may also be 
compression, transmission and high fidelity reconstruction referred to with the acronym AFLC-VQ. The selection of the 
of such data, especially image data. Although other infor- 15 number of clusters generated is decided by the quantization 
mation types may benefit from the present invention, it level to be applied i.e. whether coarse or fine quantization is 
applies with special interest to complex images such as needed. AFLC-VQ is based on an integration of self- 
medical and satellite images. organizing neural networks with fuzzy membership values 

of data samples for upgrading the centroid values of the 

BACKGROUND OF THE INVENTION 20 clusters as the samples are presented thus eliminating most 

Urge volumes of medical imagery demand high fidelity misclassifications. This technique is also noise tolerant and 

image compression for archiving and transmission. Lossy avoi ? s man y Problems associated with earlier vector quan- 

compression of medical images, however, has always been tization efforts particularly for large medical images. The 

a subject of controversy since the loss of information improved performance ofthis new technique is its capability 

involved is thought to affect diagnostic judgment. Most 25 to generate high fidelity reconstructed images even at very 

radiologists prefer not to exceed a compression ratio of 10:1 low bit rate ^ the compressed image with acceptable MSE 

when JPEG compression is used. Application of Wavelet ( mean «l uare error ) and PSNR (i*^ si S nal to noise ratio). 

Compression to digitized Radiographs, by M. A. Goldberg, Vector quantization algorithms provide advantages for 

el al., Vol. 163, pp463-468, AJR(1994) describes wavelet m g n fidelity transmission of medical images at very low bit 

based compression techniques which may not reduce the 30 rates in a fast, progressive manner for applications in tele- 

image quality even when subjected to compression ratio up medicine and other interactive web based downloading of 

to 30:1. Wavelet coding is a class of sub-band coding that large medical images in monochrome and color. In a pre- 

allows selective coding of sub-bands significant to visual ferred embodiment, the addition of an adaptive arithmetic 

perception. The encoding process actually determines the coder further improves the performance of the application of 

distortion introduced by quantization. It is well known from 35 the AFLC-VQ algorithm. An advantage of being able to use 

rate distortion theory that vector quantization yields optimal suc h low bit rate, i.e. a high compression ratio without 

distortion. Implementation of vector quantization, specifi- introducing significant distortion in the reconstructed image, 

cally the encoding process, is, however, quite involved. is that under medical emergency conditions the compressed 

However, the decoding process in vector quantization is image can be sent to a remote location quickly (since there 

relatively simple and can be achieved from look up tables. 40 are few bits) over the internet for immediate evaluation 

Vector quantization of standard images is a well-researched while images with better quality can follow in a progressive 

topic. See special Issue on Vector Quantization, IEEE Trans. manner if desired. 

Image Process (1996). Other prior art approaches, in A paper relevant to the present invention, authored by the 

general, use a simple statistical clustering technique for present inventor, Sunanda D. Mitra, along with Shuyu Yang 

generating the code vectors in vector quantization. Limita- 45 (who is not an inventor of the present invention), entitled 

tions of these earlier algorithms include long search pro- High Fidelity Adaptive Vector Quantization at very low bit 

cesses and getting trapped in local minima. Yet other rates for Progressive Transmission of Radiographic Images, 

research discloses clustering techniques for efficient code- was published in Journal of Electronic Imaging, Jan. 1999, 

book design, including fuzzy clustering in deterministic M>1. 8(1)/1. This paper is hereby incorporated herein by 

annealing associated with the materials sciences. 50 reference. 

Multi-resolution codebook design, as known in the art, For a better understanding of the present invention, 

provides a database of the centroid vectors or code vectors together with other and further objects thereof, reference is 

of differing resolutions. These centroid vectors as discussed made to the accompanying drawings and detailed descrip- 

herein are used to reconstruct the entire set of vectors in a 55 tion and its scope will be pointed out in the appended claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

art. When a specific code vector is to be sent, the index into FIG. 1 is a schematic block diagram of the inventive 

the codebook is sent instead. The receiver indexes into the Vector Quantizer; 

same codebook and retrieves the code vector being sent. In ^ FIG. 2a contains Tables 1(1) of vector quantization for a 

this way the data being sent is compressed. visible human image in red; 

However, it would be desirable to provide a system that FIG. 2b contains Tables 1(2) of vector quantization for a 

provides for image data compression and storage and/or visible human image in green; 

transmission of that compressed data and subsequent high FIG. 2c contains Tables 1(3) of vector quantization for a 

fidelity reconstruction of the image. 55 visible human image in blue; 

It is an object of the present invention to compress data by FIG. 3 is Table 2 of vector quantization for a human 

combining vector quantization and adaptive fuzzy leader cervical image; 



12/15/2003, EAST Version: 1.4.1 



US 6,421,467 Bl 

3 4 

FIGS. 4a and 4b are histogram associated with Table 1 original function f(x) in the 4>(x) sub-space. The expansion 

and Table 2; coefficients are computed from orthononnalily and vanish- 

FIG. 5a and Sb are two Tables 3(a) and 3(b) comparing m S moments criteria to ensure exact reconstruction. The 

AFLC-VQ, JPEG and EZW for different compression rates ?etaib> of muluresolution wavelet rcconstmcnon can be 

(CR), means square error (MSE) f and peak signal to noise 5 found in well known references^ Such mulu-resoluuon 

ratios }PSNRV decomposition can be extended to 2-D signals and used for 

' . image compactness and decorrelation prior to scalar or 

FIGS. 6(a) and 6(b) are graphs of algorithm performance; vector quantization. In 2-D multi-resolution discrete wavelet 

FIG. 7 has histogram comparisons of cervical spine image decomposition, the 2-D scaling function <K X > V ) is assumed to 

reconstruction; and 10 be separable, i.e., <K x >yHK x )<t>(y)* If M x ) *n d M>(y) a« 

FIG. 8 has histogram comparisons of three color human companion wavelets to scaling functions <j<x) and <j>(y), then 

image reconstruction. me three basic 2 " D wavelels mi 

^ 1 =4»(x)i|)(y), n) 2 ^(x>()(y), and H^(xMy)- 

DESCRIPTION OF PREFERRED Therefore, 2-D discrete wavelet decomposition at each 

EMBODIMENTS level yields horizontal, vertical, and diagonal edge informa- 

„ .„ * . .i. i • ■ i c i * u j 15 lion corresponding to the three basic 2-D wavelets. The 

FIG. 1 illustra.es the general pnnc.ple of wavelet based multiresolu ^ n wavelet decomposition of an image, based 

vector (puliation for encoding an image. The ongina oq ^ di^ion, * a 7ype of sub-band analysis 

input image 2 is decomposed (wavelet decomposition/ em wfaich ^ fa »d stmcmc f 

transformation filter) 4 into four levels. Each high resolution , . . ... rj . , . . , 

, . . f . , , a low level smooth image and several error images belong- 

lm age at each level is then vector quantized by AFLC-VQ 6 20 - , . 1 © » 

b . . . . . . 1 o a j- 1 uig to different resolutions. 

to generate a multiresolution codebook o. As discussed , - , , r . 

, , . j . - . * . , , -j ci „ Since a coder performance at any level of decomposition 

below, the codebook is indexed and an index file represent- . . . r , . , / , ^ .. r r 

* , . fa c j*-r j and onentation can be predicted from the statistics of the 

mg the decomposed/transformed sub-image data is formed. .. , , , .... ,. # ., • v 

* j j L 1 * . -1 j ^ sub-image and such pdf (probability distnbution function) 

The indexed codebook is sent and stored at the receiver. . . * j . i_ ■ * 1 

, ... 4 4l _ i j * l • . ■ iL has been reported to be approximately represented by a 

When data is being sent, the actual data being sent is the 25 ,- * £ c •« 1 . JC k 1 .u 

. , , , f 1 J . . ■ r™ t . , , generalized Gaussian for specific wavelets and filter lengths, 

index into the codebook for the sub -images. This index data fL , c . f , . , «, ' 

, , ft c . & ... fhree classes of wavelets were analyzed and the filter 

is entropy encoded 9 for sending via a communications 1 • a c nn m-.-n. 1 * r a 

, . ™ ■ j • j 1 . • , lengths vaned from 0 to 20 coefficients. The goal was to find 

channel 11 to a receiver. The received index data is entropy 0 , f f ... . , , , -7 , 

jji^n j.u. r it_- Ihe pdf of subimages which would best fit a generalized 

decoded 13 and the transformed sub-images are recon- ^ • c .u r n • r r w 1 r 

.jiei.il- *u j j r *u Gaussian of the following form for a specific class of 

structed .15 by looking up the received indexes from the 30 - , tU xr ui tr 1 • 

, , . 0 • • , , 1ft images, namely the Visible Human color images: 

codebook 8 at the receiver. The image is reconstructed 10 & J 0 

ready for presentation 17. PU^^K-K^) (3) 

The filter in items 4 and 10 of FIG. 1, includes a length 

of wavelet transformation 4 (and related 10) matrix which 

plays an important role on the accuracy of the encoding 35 ^ 

scheme. Based on the distribution characteristics of the J 3 yi 

wavelet coefficients, appropriate filter lengths for specific ^ = frruw ^ b = J_ IwJ 

wavelets are chosen prior to encoding. 2rf— ) ^ r/ 1 ^ 

The wavelet transformation of a signal provides a com- \ r mj) \r^j 

pact representation of the signal in the space/time frequency 40 

plane. In practice, the finite multiresolution wavelet repre- . 

sentauon of a signal can be expressed as a sum of superpo- WDere T . iS the standard Gamma ^ton-^ f 

sitions of two properly chosen orthonormal basis functions Experimental results indicate that the pdf s of the wavelet 

known as the wavelets and the scaling functions. coefficients of sub-images of typical Visible Human images 

„ .. . . . . A ... ... - 45 are represented by a generalized Gaussian function with the 

The multiresolution discrete wavelet decomposition of . r - tl _ , , 1T 

- . c/ x . 4 fi 1 ■ value of the parameter r around 0.7. However the difference 

any arbitrary function f(x) in terms of two properly chosen . r , , tl _ . , . . 4 . 4 , . . 

3 £ i - /i \ j in vanances between the experimental and the theoretical 

sets of orthonormal basis functions aMx) and <p(x) is . . . , . . r . . 4 C1 , . . 

«r\ / yv / pdf s are minimized by choosing appropriate filter lengths 

expresse as ancJ a S p ec ^ c wave ] e t family for specific classes of images. 

50 With the use of Daubl2 (a process well known in the art) 

/( X ) = £ c^q-vzJ* -tibo) + for wavelet decomposition, the number of clusters resulted 

*=->» ^ ' from AFLC vary more smoothly and show more controlla- 

t o, bility of compression ratio. 

^ ^^M^ ""^l Preferred embodiments of the present invention are best 

"* =I w=_ ~ 55 understood by describing the major component parts, Vector 

Quantization (VC) and Adaptive Fuzzy Leader Clustering 

where the scaling coefficient is the inner product (AFLC) separately. 

Vector Quantization: 

j - (2 ) ^ vector quantizer Q is a mapping from a vector in 

cun = </(<*). <>LnW) = "IT? J Hx)6{a^ L x - nbo)dx 60 k-dimensional Euclidean space, R*. into a finite set W 

containing M reproduction vectors, referred to as code 
vectors or codewords: 

and where tp(x) and <j)(x)eZ 2 are known as the mother Q: R*-*W, 

wavelets and the scaling functions respectively. It can be Where W~{w t , i«l, 2 . . . M} is the set of reproduction 

shown that the detail or information lost in representing a 65 vectors. 

function f(x) in a lower resolution by taking an inner product Vector quantization can be described as the process of 

of f(x) with the scaling function $(x) is the projection of the grouping similar vectors into the same cluster so that the 
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centroid vector of each cluster is able to reconstruct the ^^Mi/h^i . . . c . . . n, 

entire set of vectors in that cluster with a minimum distortion where is a numerical vahie in and expresses the degree to 

dictated by the similarity and optimization criteria which the element belongs to the ith cluster. A widely used 

employed. Therefore the centroid vectors or the code vectors clustering method is the fuzzy c* means (FCM) algorithm 

form the codebook that consists of the addresses of all code 5 developed by Bczdek. The objective function of the FCM 

vectors thus reducing the file size of the original data set algorithm takes the form of 
tremendously while introducing minimal distortion. The 

distortion introduced depends on the complexity of the c « 

similarity criterion, optimization criterion, and the level of Jim* ) = 2 2^'^ " Vl " 2 ' m > 1 

quantization used. In most vector quantization techniques, JQ i=l >! 

Euclidean distance measure is used as a similarity measure. 

An integrated neural-fuzzy clustering technique is used, . „ . _ # . , . . , . . . . a 

-.u j-c j aut /a j n \ . where m is called the exponential weight, which influences 

with a modified ART (Adaptive Resonance Theory) type ... rt . . rr , x 

1 *. 1 *u * • 11 1 . *u * • . • -i Ibe degree of fuzziness of the membership (partition) matrix, 

neural networks that initially clusters the vectors into similar T 1 iU - • • ■ *• « *k u- 1 

, 4 , , / t J j.jl To solve this minimization problem, the objective function 

groups, and then the cluster centroids are updated by an . rJ n * a * j-*c L . J - 4 l . . /c 

& . * . • ■ 1 j * c u t_- 1 k m equation [4] is first differentiated with respect to v £ (for 

optimization constraint including fuzzy membership values c ~? . > J . , v . tU t r /e * c v , 

r *i_ j * 1 * 11 .1. ; fiL t £ fixed . . .,c,j 1 n) and then to /*-, (for fixed 

of the data samples following the use of the fuzzy C means . \ A A 1 • a. j *- ' e 1 

/r^>w\ i- *• f *• r*i_- 1 v.. i»l, . . . . c). After applying the condition , , u lV - 1, 

(FCM) nonlinear equations. A modification of this algorithm J' i - ' r^il: y 

u j.u#* 1-1 ■ -1 u%- Vj=l,2, . . . n, equauon [4] gives 

may be used that includes a new similarity measure enabling J L J 0 

formation of clusters of any shape and a Kohonen type 

upgrading rule for the centroids. TTiese modifications allow 20 v . _ 1 y fa..y* x . t 1 = 1,2 c (5) 

better allocation of the vectors to the proper clusters by ^ii^Ti^ " ' 

introducing an intra-cluster membership fiinction while J" 1 

upgrading the centroids. It also eliminates some of the 2 ^ 

problems with ART-type networks due to normalization of ^ = ^ ( 1 = 1,2 c; j = 1, 2 n, 

the sample vectors. Although the achievability of minimum 25 V (1 - - n h 2 ) 1 "" -1 

distortion at a specific bit rate by vector quantization has £1 J 
been theoretically proven from rate distortion theory almost 

half a century ago, practical implementation of VQ for small , , . . 

sizes and classes of images has been achieved relatively ^ stem ^ e 2, uatl0n (5 > ^Ifff °" < 6 > 

recently. Many of the earlier algorithms using simple sta- 30 C3an0 \ te solved analyttcally However, the FCM algorrthm 

„ , & , f * r . provides an iterative approach to approximating the mini- 

Ustical clustering suffer from a number of problems namely mum Qf ^ objectiye startin ^ from a ^on. 

lack of convergence getting trapped in local rninima, and Neural networks draw their Aspiration ^ the function 
inability to handle large data sets. More advanced vector of me brain and mode l the brain at the level of neurons, 
quantization algorithms have eliminated some of the above Motivation for using neural models for performing intelli- 
problems. However, vector quantization of large data sets as 35 gent tas ks stems from the fact that humans exhibit remark- 
encountered in many medical images still remains a chal- a ble ability to cope with unexpected situations and to learn 
lenging problem. without the help of any teacher. Perhaps this explains why 
The present invention provides an adaptive vector quan- learning and classification are the central issues of neural 
tization technique that is capable of encoding large size as network research. Here, the concepts of two self-organizing 
well as color images with minimum distortion in the 40 neural networks which perform clustering by different 
decoded images even at a compression ratio (CR) above approaches are presented. The ideas embedded in these 
100:1. The success of this new technique depends on an neural networks have been integrated with fuzzy member- 
adaptive clustering technique with efficient optimization ship concepts to develop the AFLC and IAFC clustering 
criteria in combination with multiresolution wavelet decom- algorithms. 

position. 45 Kohonen Clustering Network (KCN), known in the art, is 
For each image class, a suitable vigilance parameter, x, an unsupervised scheme, which finds the "best" set of 
needs to be selected for a range of clusters that could be weights for hard clustering in an iterative, sequential man- 
formed to generate multi resolution codebooks for wavelet ner. The structure of KCN consists of two layers: an input 
decomposed sub images. Once the parameter x is (fan-out) layer and an output (competitive) layer. Given an 
established, the optimization process involves updating of 50 input vector, the neurons in the output layer compete among 
the cluster centroids, initially formed by the modified self themselves and the winner (whose weight vector has the 
organizing neural network architecture, by using the well minimum distance from the input vector) updates its weight 
known FCM equations. and those of some set of predefined neighbors using equa- 
In non-fuzzy "hard" clustering, the boundaries of different tion (7), below. The process is continued until the weight 
clusters are such that one pattern is assigned to exactly one 55 vectors "stabilize." In this method, a learning rate must be 
cluster. On the contrary, fuzzy clustering provides partition- defined which decreases with time in order to force termi- 
ing resulting from additional information supplied by the nation. The update neighborhood that is also reduced with 
cluster membership values indicating different degrees of time must be defined by 
belongingness. 

Consider a finite set of elements Xo{xj, . . . , x„} as 60 J*n,(*) + otA)[^(Jt)-H^(Jt)], for ie AW), otherwise (7) 

being elements of the p-dimensional Euclidean space R^. + = | ^ ^ 

The problem is to perform a partition of this collection of v 
elements into c fuzzy sets with respect to a given criterion, 

where c is a given number of clusters. Usually, the criterion for all i and j, where N t - (k) is the neighborhood set of the 

is to optimize an objective function that acts as a perfor- 65 winner node, and a(k) is the learning rate, 

mance index of clustering. The end result of fuzzy clustering A competitive learning model does not always leam a 

can be expressed by a partition matrix U such that stable code in response to arbitrary input patterns and 
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problems arise especially if too many input patterns are 

presented to the network or if the input patterns form too p (8) 

many clusters. The reason is that, as new input patterns Yismax^x^bu^lsjsN 

arrive, old patterns are washed away due to the plasticity or i=I 

instability of competitive learning model. Grossberg's 5 

Adaptive Resonance Theory (ART) known in Che an, ^ ^ ^ ^ ^ comparison 

addresses exactly this plasticity-slabdity problem and forms , t . . t , 4 • . j -*u v /♦£ 

the basis of ART-1, ART-2 and ART-3 c, " sler P' 0 ' 01 ?* 6 with Y, (the cen- 

The ART-1 architecture has two layers of neurons called *°j d v <> and ,he current ,n P ut vector *i 15 P erf ° m > ed 25 

the comparison layer (Fl) and the recognition layer (F2). ao toilows - 
The classification decision is indicated by a single neuron 

activated in the recognition layer. The neurons in the com- r = ^ X J~ V ^ < r ( 9 > 

parison layer respond to input features in the pattern (in J_ jj \\x k - Vi \\ 

ART-1, inputs are binary valued). The weights between Nil=l 

these two layers are modifiable in both directions according 15 
to two different learning rules. The recognition layer neurons 

have inhibitory connections that allow for a competition. where N <* is ^ number of s™*** m class 1 If ^ ratio R 

The network architecture can be separated into two parts: the 15 less than a user-specified threshold x, then the input 

attention and the orienting systems. The attention system has belongs to the cluster originally activated by the simple 

two layers of neurons (comparison and recognition) with 20 competition. The choice of the value of x is determined 

feed-forward and feedback characteristics. This system empirically after a number of initial runs, 

determines whether the input pattern matches one of the When the input is classified as belonging to an existing 

prototypes stored. If a match occurs, resonance is estab- cluster, it is necessary to update the prototype (centroid) and 

lished. The orienting subsystem is responsible for sensing tne bottom-up weights associated with that cluster. First, the 

mismatch between the bottom-up and top-down patterns on 25 degree of membership, /< --, of X- to the winning cluster, v., 

the recognition layer. The recognition layer response to an ^ calculated y^ing equation. If the fuzzy membership value 

input vector is compared to the original input vector through of ^ • , Uern ^ k&s ^ a lhlcshold 0> afjx rese t s 

a mechanism termed vigilance. Vigilance provides a mea- ^ ^ node and a ocedure t0 find a new 

sure of the distance between the mput vector and the cluster - . * . ( . , lt • 4 

j . . . * * j . , L winning node whose centroid is closest to the mput pattern, 

center corresponding to the activated neuron m the recog- 30 IJ?4 . J* . . ~. . 4 „ . 

nition layer. When vigilance falls bellow a preset threshold, l ! ^ ^ meml > ersh *P value of the input pattern is greater 

a new category must be created and the input vector must be than CT ^ e new duster centroid for the winning node Yi is 

stored into that category. That is, a previously unallocated u P dated b * ec 5 uatlon where Parameter n changes to N, (the 

neuron within the recognition layer is allocated to a new number of samples m cluster i). The AFLC model was later 

cluster category associated with the new input pattern. 35 modified to develop the IAFC model to eliminate some of 

The recognition layer follows the winner-take-all para- the inherent problems in the ART-rype algorithms. A brief 

digm (also known as MAXNET). If the input vector passes description of the IAFC model is provided below, 

the vigilance, the winning neuron (the one most like the An IAFC model, known in the art, introduces a similarity 

input vector) is trained such that its associated cluster center measure for the vigilance criterion incorporating a fuzzy 

in feature space is moved toward the input vector. 40 membership value to Euclidean distance as follows: 

Neuro-Fuzzy algorithms retain the basic properties and 

architectures of neural networks and simply "fuzzify" some R-e-^jpc-vJl (io) 
of their elements. In these classes of networks, a crisp 

neuron can become fuzzy and the response of the neuron to wnere ^ ^ lne fuzzy membership value of the input data 

its activation layer signal can be of a fuzzy relation type 45 pattern m me ith ^^ng cluster, and y is a multiplicative 

rather than a continuous or crisp function type. Examples of factQr that lhe sh of the dusters Such M 

this approach can be found where domain knowledge in ation oonsidei5 not onl the Euclidean dislance 

becomes formalized m terms of fuzzy sets and afterward can . , - , . j r ■ . 

, v j * u *u i i *i_ fl i i between the mput pattern: and the centroid of a winner but 

be applied to enhance the learning algorithms of the neural , * • 

networks or augment their interpretation capabilities. Since 50 ako * c relat ! ve V , ^1 ™c * ^ 

the neural architecture is conserved, changes are made in the currenU y f*?^ cluster centr0lds 45 the de 8 ree of sim ^ T ~ 

weights connecting the lower layer to the upper layer "y. This similarity measure can cause the cluster centra ids to 

neurons arcmn d preventing weights from converging quickly. 

AFLC is a hybrid neuro-fuzzy system that can be used to n * incorporation of an intra-cluster membership value and 

learn cluster structure embedded in complex data sets, in a 55 a function of the number of iterations into a Kohonen-type 

self-organizing, stable manner. A choice of the Euclidean learning rule eliminate this problem, 

metric is made in developing the AFLC system while IAFC finds a winning cluster by performing the dot 

keeping a simple control structure adapted from ART-1. The product (similar to ART and AFLC) between the normalized 

AFLC algorithm initially starts with the number of clusters input vector I and the bottom-up weights b,-. The neuron that 

set to zero. The system is initialized with the input of the first 60 receives the largest input activation Y, wins the competition, 

feature vector X ; . As is to leader clustering, this first input is After selecting a winning cluster, the IAFC algorithm per- 

said to be the prototype of the first cluster. The next forms the vigilance test according to the vigilance criterion 

normalized input feature vector is then applied to the between the unnormalized input vector X and the winning 

bottom-up weights in a simple competitive learning scheme, node. If the winning cluster satisfies the vigilance test (R<x), 

or dot product. The node that receives the largest input 65 the centroid is updated using the following equation: 
activation Y ( . is chosen as the prototype vector. As in the 

original ART-1: v^^i-A^K^+x^x (n) 
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where, The related measure of PSNR (in dB) is computed using 



and 



°NMSE *j 



;r(X; v?* 0 , r) = < I3 > The bit per pixel (bpp) value and compression ratio (CR) are 

. v2 lBl ~ calculated as follows. 

1-2(|]X -v^H/r) 2 , when 0 s ||X - v^U s r/2 

2(1 _ [|* - / r) 2 , when r/2s [|X - vf^H s: 10 . Number of bits required to represent the image ( 17) 



(flW1 „ Number of pixels in the image 

0, when \\X - vf^'U * r K ^ 



and 



Total Number of bits of original image (18) 

The it-function decides the intra-cluster membership value cr = Total Number of bits of imR&c 

of the input data pattern in the ith-winning cluster, where 1 15 . c . . . . 

. , r , -T . ... . ° bpp of original image 

is the number or iterations and k is a given constant. = r r —. 

0 bpp of compressed image 

The IAFC algorithm initially starts with the number of 
clusters set to zero. When the first input pattern is presented, 

it becomes the first cluster centroid. For the next input Experimental Examples: 

pattern, the membership value will obviously be one since 20 Th e wavelet transform of an image decomposes the image 

only one cluster is present. In this case, the vigilance mt0 different subimages containing different information, 

criterion used is just ||X-vJ<r, as the fuzzy membership Subimages in higher levels of decomposition contain more 

value has no physical meaning. If it satisfies the vigilance formation while lower level subimages contain moredetail 

test, the centroid is updated by the statistical average of the m ^ >■ Expenmental results demonstrate that, when 

between two points. After more than one cluster are formed, 25 f n ^ *J* COm P OSe lDt0 , 4 leve ^ subimages on the 1" 

the IAFC algorithm uses the fuzzy membership value for the level ™* 2 . * evel can b f selectively discarded or coarsely 

vigilance test and the intra-cluster membership value for quantized without severely deteriorating the visual effect of 

updating the centroid of a winning cluster. While determin- the reconstructed image. 

ing a winning cluster, the winner is decided by the direction Codebook size resulting from AFLC-VQ depends mainly 
between the input data and the centroid of clusters. This 30 00 the use^specified vigilance test parameter x (tao, the 
procedure can cause misclassifications because a cluster for symbol and the word are used interchange ab y herein) 
which the centroid has the smallest difference of angle wins whlt ; h ha f a ^ * m ™ b * r < C ) 
the competition even though it is located farther from the fr ° m algorithm. FIG. 2 m Tables 1(1) 1(2), and 
input data than other cluster centroids. To avoid 1<3) give the x-C z^lte denv^ from a 2048x1024 Visible 
misclassifications, the IAFC algorithm calculates the mem- 35 Human Image a^l^ and FIG. 3, Table 2 show the x-C 
bership value after selecting the winning cluster. If the fuzzy ^sults from a a typical 512x512 image for the Cervical 
membership value is less than a threshold a, decided by the S P me tom u on Z m *} ff»P^ Vtstok Human images 
user, the IAFC algorithm activates the procedure to find the *f 030 te «* n from fables of FIGS. 2 and 3, lower 
cluster that is closest to the input pattern. TT,e closest cluster ^solution subimages can be quantized in a smoother con- 
becomes the new winner. 40 Unuous manner than the hi e her level subimages. For low 

„ . . . . „ , c . . compression ratio, both the lower level and higher level 

Suitable criteria, also called performance criteria, are . . , a , , ... . « i 

, . . i . .1. r r subimages are quantized finely by specifying a small value 

needed to rigorously evaluate the performance of a com- c . . . J J . c J ' r ... 

, 5 . 7 c p ' ^ c . , of x so as to preserve as much image information as possible, 

pression scheme. In case of images, the search for sun pie ~ v . , f . . . ° t . r , 

j ui • * l* j « . c ,i * *l On the other hand, for high compression ratio, high- 

and suitable cntena is hindered by the fact that the results AC , .. u i *• i j- j a I7 

3 . 45 resolution sub images can be selectively discarded or roughly 

obtamed by statistical performance cntena may not agree j u -j* 1 i u-.Tt. 

• iL i_- f c x. x. o- . quantized by providing a large x value combining together 

with the subjective evaluation of the human eye. Since the ... • . i ji i ■• 

...... . J . with an appropriately quantized lower resolution subimages. 

objective is data compression, the compression ratio at ~ . l ■ .u » o r .u * ui * i j ™^ 

J .. * j ■ . • u - , : t r Graphs showmg the tao-C of the tables in FIG. 2 and FIG. 

optimal distortion is obviously an important performance , , . rip . - 

r ™ « . *. T , . 3 are shown in FIG. 36 and 3c. 

measure. The reconstructed image quahty can be evaluated n lt . . . 

, /fcfoF-\ .i. * - j so Results according to the inventive compression scheme 

using the mean-squared error (MSE), the normalized mean- . . . A • ^ . , ^ A ™ u 

j /xrwe^x . • . . . . are presented in FIG. 4 in Tables 3(a) and 3(o). fhese results 

squared error (NMSE) and peak signal-to-noise ratio u t , . A17I ^. j j . * j - 

2, vm v v , • tu . • j * i s h° w that AFLC-VQ compressed and reconstructed images 

(PSNR) as error metrics. These error metrics do not always . ... Kjfcr: , n nvm t u mrr c u- u * 

v . ; n . . , . .. . . , . J show better MSE and PSNR than JPEG for the highest 

correlate well with perceived image quality, although they # - , T . . , . * 

•j ..... , r i f ■ i-. t>, compression ratio used. It is noteworthy, however, that the 

provide statical measures of relative qual.ly. Denotmg the nlio led for ^ ^ al rithn] does 

onginal NxN unage gray level matnx by f and the earn- *^ ^ » 

sponding reconstructed image matnx by f, MSE and NMSE At7T ^> u 6 . 4 . ,. 6 f 

u . When AFLC-VQ results are compared with the results of 

are given by. EZW without entropy coding, the performance of AFLC- 

v lM t (14) V Q is much better than EZW in terms of MSE (mean square 

mse= — V Y [f(U j)-TiU j)] 1 . and 60 error ) ^ PSNR(peak signal to noise ratio). An adaptive 

^ 2 £o >o arithmetic coder for the multiresolution codebook generated 

by AFLC-VQ is currently under development. The addition 

l £A ^ r/(/ ( »_7(i, j)] 2 (l5) of the entropy coder to the current compression scheme is 

nmse= — —p ■ expected to result in better performance of AFLC-VQ over 

i-o /-o "* 65 JPEG for all compression ratios. FIGS. 5(a) and 5(b) show 

graphical representations of the performance for all the 
algorithms used. Actual examples of cervical spine image 
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reconstructed by AFLC-VQ are visually superior over the 
ones by JPEG, and EZW with and without entropy coding at 
compression ratios around 100:1. The degradation in visual 
appearance of the images reconstructed by JPEG and EZW 
does not correlate with the statistical performance criteria 
such as MSE and PSNR as documented in other current 
literature. AFLC-VQ out-performs JPEG and EZW whose 
reconstructed images show deterioration dominated by the 
blocking artifact. FIG. 6 (c)-(0 shows the histograms of the 
above reconstructed images. It can be seen that compared 
with EZW and JPEG, AFLC-VQ reconstructed images pre- 
serve a much better shape of histogram of the image than 
those reconstructed from EZW and JPEG. Histograms of 
three color planes (R, G, B) are shown in FIGS. 7 (a)-{/i). 
The high fidelity to the original image histogram by AFLC- 
VQ reconstruction in all color planes gives a sharp contrast 
to the much-deteriorated histograms from both EZW and 
JPEG reconstructions. 

Application of fuzzy type clustering in nonconvex opti- 
mization problem in the framework of statistical mechanics 
has been addressed by maximizing the entropy at a given 
variance. The present invention provides for applying fuzzy 
clustering to codebook design leading to an efficient global 
optimum. The use or neuro-fuzzy algorithms for efficient 
vector quantization have led to high fidelity reconstruction 
of medical images even at very low bit rates. 

Although the invention has been described with respect to 
various embodiments, it should be realized this invention is 
also capable of a wide variety of further and other embodi- 
ments within the spirit and scope of the appended claims. 

What is claimed is: 

1. A method for sending an image over a communications 
channel between and sender and a receiver comprising the 
steps of: 

decomposing said image into sub -image data by multi- 
resolution wavelet decomposition/transformation, 

vector quantizing said sub-image data, 

vector quantization processing said vector quantized data 
with an adaptive fuzzy leader clustering algorithm, 

forming an indexed multi-resolution codebook and stor- 
ing said codebook at both the sender and the receiver, 

sending said processed and quantized sub-image data as 
an index file of said multi-resolution codebook, 

receiving said index file and retrieving said processed and 
quantized sub-image data from said multi-resolution 
codebook at the receiver, and 

reconstructing said image data by inverse wavelet 
decomposition/transformation, and 

presenting said reconstructed image. 

2. The method as defined in claim 1 further comprising the 
steps of: 

forming the indexed multi-resolution codebook at the 
sender, 

sending said indexed codebook to said receiver, and 
storing said indexed codebook at said receiver. 

3. The method as defined in claim 1 further comprising the 
steps of entropy encoding of the index file being sent and 
entropy decoding of the index file being received. 

4. The method as defined in claim 1 wherein the step of 
vector quantizing comprises the steps of: 

generating vectors from said wavelet transformed sub- 
image data, 
grouping similar vectors, and 

forming a centroid vector of said groupings of similar 
vectors. 
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5. The method as defined in claim 4 further comprising the 
steps of: selecting a vigilance parameter x, said vigilance 
parameter x determined by the ratio R, 

using R to group said vectors, where Nj is the number of 
5 samples in class i, and X ; is the current input vector, and 
Vj is the winning centroid vector, and wherein 

6. The method as defined in claim 1 wherein said step of 
decomposing by multiresolution wavelet transforms 

15 includes the step of decomposing said image into a plurality 
of levels, wherein each level has a different resolution of said 
image, and said step of reconstructing said image includes 
reconstruction of said plurality of levels. 

7. The method as defined in claim 1 wherein the step of 
20 decomposing comprises the steps of: providing a filter 

function and choosing filtertaps. 

8. The method as defined in claim 7 wherein said step of 
proving a filter functions comprises of the step of selecting 
an optimum filter length for the wavelet transformation for 

25 optimum representation of the image data. 

9. A system for sending an image from a sender to a 
receiver over a communications channel comprising: 

means for decomposing said image into sub-image data 
by multiresolution wavelet decomposition/ 
30 transformation, 

a vector quantizer operating on said sub-image data, 

means for processing said vector quantized data with an 
adaptive fuzzy leader clustering algorithm, 

an indexed multi -resolution codebook stored at both the 
35 sender and the receiver, 

means for sending said processed and quantized sub- 
image data as an index file of said multi-resolution 
codebook, 

40 means for receiving said index file, 

means for retrieving said processed and quantized sub- 
image data from said multi-resolution codebook at the 
receiver, and 

means for reconstructing said image data by inverse 
45 wavelet decomposition/transformation, and 
means for presenting said reconstructed image. 

10. The system as defined in claim 9 further comprising: 
means for forming the indexed multi-resolution codebook 

at the sender, 

50 means for sending said indexed multi-resolution code- 
book to said receiver, and 
means for storing said indexed multi-resolution 
codebook at said receiver. 
55 11. The system as defined in claim 9 further comprising 
means for entropy encoding of the index being sent and 
means for entropy decoding of the index being received. 

12. The system as defined in claim 9 wherein said means 
for vector quantizing comprises: 
60 means for generating vectors from said wavelet trans- 
formed sub-image data, 
means for grouping similar vectors, and 
means for forming a centroid vector of said groupings of 
similar vectors. 

65 13. The system as defined in claim 12 further comprising: 
means for selecting a vigilance parameter x, said vigilance 
parameter x determined by the ratio R, 
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means for using R to group said vectors, where N is the 
number of samples in class i, and X is the current input 
vector, and v is the winning centroid vector, wherein 



llX,--V;il 
R = ~ < T. 



14. The system as defined in claim 9 wherein said means 
for decomposing by multiresolution wavelet transforms 
includes means for decomposing said image into a plurality 
of levels, wherein each level has a different resolution, and 
said means for reconstructing said image includes means for 
reconstruction of said plurality of levels. 

15. The system as defined in claim 9 further comprising 
means for filtering and means for choosing the filtertaps for 
optimal wavelet decomposition/transformation. 

16. The system as defined in claim 15 wherein said means 
for choosing comprises means for selecting an optimal filter 



10 



15 



14 



length for the wavelet transformation for optimal represen- 
tation of the image data. 

17. A method for sending an image over a communica- 
tions channel between and sender and a receiver comprising 
the steps of: 

decomposing said image into sub-image data by multi- 
resolution wavelet decompositionAransfonnation, 

vector quantizing said sub-image data, 

vector quantization processing said vector quantized data 
with an adaptive fuzzy leader clustering algorithm, 

sending said processed and quantized sub-image data, 

receiving said processed and quantized sub-image data at 
the receiver, and 

reconstructing said image data by inverse wavelet 
decomposition/transformation, and 

presenting said reconstructed image. 
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